RESUMO
OBJECTIVES: Evaluate and compare the ability of large language models (LLMs) to diagnose various ailments in otolaryngology. METHODS: We collected all 100 clinical vignettes from the second edition of Otolaryngology Cases-The University of Cincinnati Clinical Portfolio by Pensak et al. With the addition of the prompt "Provide a diagnosis given the following history," we prompted ChatGPT-3.5, Google Bard, and Bing-GPT4 to provide a diagnosis for each vignette. These diagnoses were compared to the portfolio for accuracy and recorded. All queries were run in June 2023. RESULTS: ChatGPT-3.5 was the most accurate model (89% success rate), followed by Google Bard (82%) and Bing GPT (74%). A chi-squared test revealed a significant difference between the three LLMs in providing correct diagnoses (p = 0.023). Of the 100 vignettes, seven require additional testing results (i.e., biopsy, non-contrast CT) for accurate clinical diagnosis. When omitting these vignettes, the revised success rates were 95.7% for ChatGPT-3.5, 88.17% for Google Bard, and 78.72% for Bing-GPT4 (p = 0.002). CONCLUSIONS: ChatGPT-3.5 offers the most accurate diagnoses when given established clinical vignettes as compared to Google Bard and Bing-GPT4. LLMs may accurately offer assessments for common otolaryngology conditions but currently require detailed prompt information and critical supervision from clinicians. There is vast potential in the clinical applicability of LLMs; however, practitioners should be wary of possible "hallucinations" and misinformation in responses. LEVEL OF EVIDENCE: 3 Laryngoscope, 2024.
RESUMO
PURPOSE: Preoperative risk stratification for patients undergoing metastatic brain tumor resection (MBTR) is based on established tumor-, patient-, and disease-specific risk factors for outcome prognostication. Frailty, or decreased baseline physiologic reserve, is a demonstrated independent risk factor for adverse outcomes following MBTR. The present study sought to assess the impact of frailty, measured by the Risk Analysis Index (RAI), on MBTR outcomes. METHODS: All MBTR were queried from the National Inpatient Sample (NIS) from 2019 to 2020 using diagnosis and procedural codes. The relationship between preoperative RAI frailty score and our primary outcome - non-home discharge (NHD) - and secondary outcomes - complication rates, extended length of stay (eLOS), and mortality - were analyzed via univariate and multivariable analyses. Discriminatory accuracy was tested by computation of concordance statistics in area under the receiver operating characteristic (AUROC) curve analysis. RESULTS: There were 20,185 MBTR patients from the NIS database from 2019 to 2020. Each patient's frailty status was stratified by RAI score: 0-20 (robust): 34%, 21-30 (normal): 35.1%, 31-40 (very frail): 13.9%, 41+ (severely frail): 16.8%. Compared to robust patients, severely frail patients demonstrated increased complication rates (12.2% vs. 6.8%, p < 0.001), eLOS (37.6% vs. 13.2%, p < 0.001), NHD (52.0% vs. 20.6%, p < 0.001), and mortality (9.9% vs. 4.1%, p < 0.001). AUROC curve analysis demonstrated good discriminatory accuracy of RAI-measured frailty in predicting NHD after MBTR (C-statistic = 0.67). CONCLUSION: Increasing RAI-measured frailty status is significantly associated with increased complication rates, eLOS, NHD, and mortality following MBTR. Preoperative frailty assessment using the RAI may aid in preoperative surgical planning and risk stratification for patient selection.